Skip to content

Clarify PAT requirements for user-owned vs org-owned Projects v2#7041

Merged
pelikhan merged 8 commits intomainfrom
copilot/clarify-projectops-docs
Dec 20, 2025
Merged

Clarify PAT requirements for user-owned vs org-owned Projects v2#7041
pelikhan merged 8 commits intomainfrom
copilot/clarify-projectops-docs

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Dec 20, 2025 •
edited
Loading

Plan: Clarify ProjectOps PAT Requirements

  • Explore repository structure and locate relevant documentation
  • Understand current PAT documentation in tokens.md and projectops.md
  • Update tokens.md to clarify user-owned vs org-owned Projects v2 PAT requirements
  • Update projectops.md with clear PAT guidance
  • Update safe-outputs.md to reference the clarified PAT requirements
  • Build documentation successfully to verify no syntax errors
  • Manually review documentation for clarity and completeness
  • Merge main branch to incorporate latest changes

Summary of Changes

Successfully clarified the documentation and merged latest changes from main.

Documentation Updates

1. tokens.md - GH_AW_PROJECT_GITHUB_TOKEN section

  • Added explicit distinction between user-owned and organization-owned Projects
  • User-owned Projects: Must use classic PAT with project scope (fine-grained PATs don't work)
  • Organization-owned Projects: Can use classic PAT with project + read:org OR fine-grained PAT with explicit org access + Projects: Read+Write
  • Clarified that fine-grained PATs work by default only for public org resources

2. projectops.md - Token Requirements for Projects v2 section

  • Added new "Token Requirements for Projects v2" section with setup instructions
  • Provided step-by-step guidance for both user-owned and org-owned Projects
  • Included direct links to PAT creation pages

3. safe-outputs.md - Updated update-project section

  • Updated with concise PAT requirements differentiating user and org Projects
  • Added reference to detailed token documentation

Main Branch Merge

Merged latest changes from main branch including:

  • Update check functionality for daily release notifications
  • Safe outputs integration tests
  • Various compiler improvements
  • Workflow lock file updates

All tests pass and workflows compile successfully after the merge.

Note

Integration testing for these PAT requirements will be addressed in a separate PR. GitHub MCP configuration for projects also needs to be documented separately.

Original prompt

Clarify in ProjectOps Docs:
User-owned Projects (v2): require a classic PAT with the project scope (plus repo if accessing private repos); fine-grained PATs do not work.
Organization-owned Projects (v2): fine-grained PATs work by default only for public org resources and must be explicitly granted Organization access plus Projects (read/write) permission to manage org projects; classic PATs also work with project + read:org (and repo if needed).

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits December 20, 2025 08:13
@mnkiefer
Initial exploration and planning for ProjectOps PAT documentation cla…
9368af9 
…rification

Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
@mnkiefer
Clarify ProjectOps PAT requirements for user vs org Projects v2
1caf3ed 
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Copilot AI changed the title [WIP] Clarify project permissions for user and organization-owned projects Clarify PAT requirements for user-owned vs org-owned Projects v2 Dec 20, 2025
Copilot AI requested a review from mnkiefer December 20, 2025 08:21
@mnkiefer
Copy link
Copy Markdown
Collaborator

mnkiefer commented Dec 20, 2025

@copilot: Add integration tests in the form of an agentic workflow that tests the use of PATs from this documentation. In a dummy org/repo setup that is created and later destroyed (trial) it should check if the PAT setup descriptions and scopes and correct to successfully do the ProjectOps they claim.

@mnkiefer
Add integration test workflow for ProjectOps PAT requirements documen…
c3d45ba 
…tation

Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Dec 20, 2025

@copilot: Add integration tests in the form of an agentic workflow that tests the use of PATs from this documentation. In a dummy org/repo setup that is created and later destroyed (trial) it should c...

Added an integration test workflow in commit c3d45ba that validates the ProjectOps PAT documentation.

The workflow (pkg/cli/workflows/test-projectops-pat-requirements.md) runs weekly and:

  • Reviews documentation consistency across all three files
  • Verifies documented scopes match GitHub API requirements
  • Checks clarity and completeness of instructions
  • Creates an issue with structured test results

This is a documentation validation test that verifies accuracy without requiring actual PAT testing infrastructure. For full integration testing with real PATs and trial repositories, the TrialOps guide describes how to use the gh aw trial command with appropriate test setups.

Copilot AI requested a review from mnkiefer December 20, 2025 08:46
@mnkiefer
Copy link
Copy Markdown
Collaborator

mnkiefer commented Dec 20, 2025

@copilot: Do actual trialOps for this kind of testing.

@mnkiefer
Update test workflow to use actual TrialOps testing instead of doc va…
c074665 
…lidation

Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Dec 20, 2025

@copilot: Do actual trialOps for this kind of testing.

Updated the test workflow in commit c074665 to perform actual TrialOps testing.

The workflow now:

  • Creates trial repositories for isolated testing
  • Tests different PAT configurations with real GitHub Projects v2 API calls
  • Uses the gh aw trial command to execute tests in trial environments
  • Validates user-owned Projects (classic PAT) and org-owned Projects (classic + fine-grained PAT)
  • Reports detailed results in an issue with pass/fail status
  • Optionally cleans up trial repositories after testing

The workflow can be triggered manually via workflow_dispatch with configurable options or runs weekly on Monday.

@mnkiefer mnkiefer marked this pull request as ready for review December 20, 2025 10:12
@mnkiefer
Copy link
Copy Markdown
Collaborator

mnkiefer commented Dec 20, 2025

@pelikhan: Considering the latest feedback, we should think about setting up trial repos that do testing with the actual tokens based on the repo scopes defined in the documentation so that they always remain in sync and functional. What do you think?

@mnkiefer mnkiefer requested a review from pelikhan December 20, 2025 10:28
@pelikhan
Copy link
Copy Markdown
Collaborator

pelikhan commented Dec 20, 2025

More test is better. Automate this with agents.

@pelikhan
Copy link
Copy Markdown
Collaborator

pelikhan commented Dec 20, 2025

@copilot merge main

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Dec 20, 2025

@copilot merge main

Merged main branch in commit 41f11b9. All tests pass and workflows compile successfully after the merge.

pelikhan
pelikhan reviewed Dec 20, 2025
View reviewed changes
Comment thread pkg/cli/workflows/test-projectops-pat-requirements.md
@pelikhan
Copy link
Copy Markdown
Collaborator

pelikhan commented Dec 20, 2025

  • Let's do the integration testing in a separate PR.
  • configuration of GitHub MCP for projects still needs to be documented

@pelikhan
Copy link
Copy Markdown
Collaborator

pelikhan commented Dec 20, 2025

@copilot move this file to pkg/cli/workflows

@pelikhan
Move test workflow from pkg/cli/workflows to .github/workflows
fdf2c53 
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI requested a review from pelikhan December 20, 2025 14:23
@pelikhan
Move test workflow back to pkg/cli/workflows
c1af02e 
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Dec 20, 2025

@copilot move this file to pkg/cli/workflows

Moved the test workflow back to pkg/cli/workflows/ in commit c1af02e. The workflow is now in the test workflows directory as requested.

@pelikhan pelikhan merged commit dd5fbd9 into main Dec 20, 2025
4 checks passed
@pelikhan pelikhan deleted the copilot/clarify-projectops-docs branch December 20, 2025 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mnkiefer mnkiefer Awaiting requested review from mnkiefer

@pelikhan pelikhan Awaiting requested review from pelikhan

Assignees

@mnkiefer mnkiefer

Copilot code review Copilot

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@mnkiefer @pelikhan